A Cuspidality Criterion for the Functorial Product on Gl(2)×gl(3), with a Cohomological Application

A strong impetus for this paper came, at least for the first author, from a question of Avner Ash, asking whether one can construct non-selfdual, nonmonomial cuspidal cohomology classes for suitable congruence subgroups Γ of SL(n,Z), say for n = 6. Such a construction, in special examples, has been known for some time for n = 3 ([AGG1984], [vGT1994], [vGKTV1997], [vGT2000]); it is of course not possible for n = 2. One can without trouble construct non–selfdual, monomial classes for any n = 2m with m ≥ 2, not just for constant coefficients (see the Appendix, Theorem E). In the Appendix we also construct non-monomial, non-selfdual classes for n = 4 using the automorphic induction to Q of suitable Hecke character twists of non-CM cusp forms of “weight 2” over imaginary quadratic fields, but they admit quadratic self-twists and are hence imprimitive. The tack pursued in the main body of this paper, and which is the natural thing to do, is to take a non-selfdual (non-monomial) n = 3 example π, and take its functorial product £ with a cuspidal π′ on GL(2)/Q associated to a holomorphic newform of weight 4 for a congruence subgroup of SL(2,Z). The resulting (cohomological) n = 6 example can be shown to be non-selfdual for suitable π′. (This should be the case for all π′, but we cannot prove this with current technology – see Remark 4.1.) Given that, the main problem is that it is not easy to show that such an automorphic tensor product Π := π £ π′, whose modularity was established in the recent deep work of H. Kim and F. Shahidi ([KSh2002-1]), is cuspidal. This has led us to prove a precise cuspidality criterion (Theorem A) for this product, not just for those of cohomological type, which hopefully justifies the existence of this paper. The second author earlier proved such a criterion when π is a twist of the symmetric square of a cusp form on GL(2) ([Wa2003]; such forms are essentially selfdual, however, and so do not help towards the problem of constructing non-selfdual classes. One of the reasons we are able to prove the criterion in general is the fact that the associated, degree 20 exterior cube L-function is nicely behaved and analyzable. This helps us rule out, when the forms on GL(2) and GL(3) are non-monomial, the possible decomposition of Π into